Appl. No. 09/818,427 

Amdt. dated February 13, 2008 

Reply to Office Action of August 20, 2007 

This listing of claims will replace all prior versions, and listings, of claims in the 
application: 

Listing of Claims: 

Claim 1-12 (Canceled). 

Claim 13 (Currently Amended): A method for augmented reality guided 
instrument positioning, comprising the steps of: 

rendering at least one graphics path guide for indicating a path for a real 
instrument to follow to a target located inside or on the surface of a real object, 
the graphics path guide being constructed in a way that the graphics path frame 
guide frames the path outside of the real object so as not to obstruct a view of a 
central part of the real instrument along the real instrument's axis as viewed from 
the side if the real instrument is in correct alignment to said path; 

displaying the rendered at least one graphics path guide on a display 
overlaid onto a direct optical or video view that comprises a real view of said real 
instrument and the real object; 

a user performing the following steps: 

moving said real instrument, observed in said real optical or video 
view, to align the real instrument it with the at least one graphics path 
guide, observed on the display that shows the at least one graphics guide 
overlaid onto said view of said real instrument and said real object; 

aligning the real instrument with the path by determining when the 
at least one graphics path guide frames the path so that a view of a 
central part of the real instrument is not obstructed by the at least one 
graphics path guide; and 

moving said real instrument along the path so that a front portion of 
said real instrument is inserted into the object until the real instrument's tip 
reaches said target and concurrently monitoring the correct alignment 
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between the visible part of the real instrument outside of the real object 
and the framing path guide. 

Claims 14-15 (Canceled). 

Claim 16 (Previously Presented): A method for augmented reality guided 
instrument positioning, comprising the steps of: 

defining a target point on or within a real object; 

defining a path to reach the target point with a real instrument; 

rendering a graphical representation to mark the path in the form of at 
least one graphical axis marker to be viewed in the direction of the path towards 
the target on a display overlaid onto a direct optical or video view of a real scene 
that comprises said real instrument, and said real object, so that a user can 
perceive said real instrument and said at least one graphical axis marker and 
their spatial relationship in an augmented reality view, said user performing the 
following steps: 

aligning said user's augmented reality line-of-sight with the at least 
one graphical axis marker so that said path to reach said target point with 
the real instrument is aligned along said augmented reality line-of-sight; 

aligning the real instrument to the path by aligning the real 
instrument with the augmented reality line-of-sight towards the target 
point; 7 and 

moving the real instrument along the path towards the actual target 
point while keeping the real instrument it aligned with the augmented 
reality line-of-sight. 
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Claim 17 (Previously Presented): The method according to claim 16, further 
comprising the step of designing the real instrument to include at least one 
physical axis marker for alignment with said at least one graphical axis marker. 

Claim 18 (Previously Presented): The method according to claim 16, further 
comprising the step of adding at least one physical axis marker to a structure of 
the real instrument for alignment with said at least one graphical axis marker. 

Claim 19 (Cancelled). 

Claim 20 (Previously Presented): The method according to claim 16, wherein the 
at least one identified graphical axis marker has a circular shape, and is centered 
on the axis of the path. 

Claim 21 (Previously Presented): The method according to claim 16, wherein the 
at least one identified graphical axis marker is a cross comprised of an 
intersection of at least two lines, the intersection to be centered on the axis of the 
real instrument for correct alignment. 

Claim 22 (Previously Presented): The method according to claim 16, wherein the 
at least one identified graphical axis marker comprises at least two axis markers 
for controlling alignment of the real instrument along a line of sight. 

Claim 23 (Currently Amended): A method for virtual reality guided instrument 
positioning, comprising the steps of: 

defining a real target point on or within a real object; 

defining a path to reach the real target point on or within said real object; 
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tracking a pose of a real instrument with respect to a pose of the real 

object; 

rendering a graphical representation of the real instrument and the path to 
obtain a virtual instrument and a graphical virtual guide on a display, the 
graphical representation being rendered with respect to a virtual viewpoint 
chosen so that from which a virtual line of sight is viewed in the direction of the 
virtual path towards the target point coincides w i th a virtual path for the virtual 
instrument to follow during a positioning of the real instrument to the real target 
point, the graphical virtual guide corresponding to the path, the virtual instrument 
comprising a 3D structure for line of sight alignment, the 3D structure comprising 
a plurality of markers centered on and distributed along an axis of the virtual 
instrument; 

aligning the virtual instrument along the virtual line of sight according to 
the graphical virtual guide in order to accordingly align the real instrument along 
the path; and 

moving the real instrument by a user in response to viewing the virtual 
instrument and said graphical virtual guide on the display, the real instrument 
moving along the path towards the real target point keeping the correct alignment 
by observing and keeping in alignment with the virtual instrument and graphical 
virtual guide. 

Claim 24 (Canceled). 

Claim 25 (Previously Presented): The method of claim 23, wherein the virtual 
guide has a circular shape. 

Claim 26 (Original): The method of claim 25, wherein the circular shape is a ring. 
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Claim 27-28 (Canceled). 

Claim 29 (Previously Presented): The method of claim 23, wherein said plurality 
of markers comprise at least two rings, centered on an axis of the virtual 
instrument. 

Claim 30 (Original): The method of claim 29, wherein the at least two rings have 
different diameters. 

Claim 31 (Previously Presented): The method of claim 23, wherein the step of 
aligning the virtual instrument further comprises the step of choosing an 
orientation of the graphical representation around the virtual line of sight 
according to a pose of a user with respect to the real target point. 

Claim 32 (Previously Presented): The method of claim 31 , further comprising the 
step of determining the orientation such that east, west, north, and south 
correspond to right, left, forward, and backward, respectively, for the pose of the 
user in which the user faces the real target point, said determining step based on 
a selection. 

Claim 33 (Original): The method of claim 31 , wherein the orientation is 
dynamically adjusted according to a change of the pose of the user. 

Claim 34 (Original): The method of claim 32, wherein the selection is dynamically 
adjusted with respect to the pose of the user. 

6 



Appl. No. 09/818,427 

Amdt. dated February 13, 2008 

Reply to Office Action of August 20, 2007 



Claim 35 (Previously Presented): The method of claim 23 wherein the rendering 
step further comprises the step of rendering graphical information about a 
distance between the real instrument and the real target point, the graphical 
information about the distance being overlaid onto the graphical representation. 

Claim 36 (Canceled). 

Claim 37 (Previously Presented): The method of claim 23 wherein the graphical 
virtual guide and the virtual instrument are designed such that information 
corresponding to the distance between the real instrument and the real target 
point can be directly observed from an alignment of the graphical virtual guide 
and the virtual instrument. 

Claim 38 (Original): The method of claim 35, wherein said rendering step is 
performed according to a virtual camera with a wide angle lens. 

Claim 39 (Previously Presented): The method of claim 37, wherein the graphical 
virtual guide and the virtual instrument are each comprised of at least one ring 
centered on the real target point respectively on the axis of the real instrument, 
and a diameter of the at least one ring is dimensioned to achieve a pre-defined 
configuration together with the real instrument when the real instrument reaches 
the real target point. 

Claim 40 (Previously Presented): The method of claim 23, wherein the graphical 
representation from the virtual viewpoint is combined with another graphical 
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representation from another virtual viewpoint looking at the graphical virtual guide 
from a side thereof. 

Claim 41 (Original): The method of claim 23, wherein said graphical 
representation from the virtual viewpoint is combined with an augmented reality 
view. 

Claim 42 (Previously Presented): The method of claim 16 wherein the at least 
one e graphical axis marker is designed such that information corresponding to 
the distance between the real instrument and the real target point can be directly 
observed from an alignment of the real target point and the at least one graphical 
axis marker. 

Claim 43 (Previously Presented): The method of claim 42, wherein the graphical 
axis marker comprises at least one ring centered on the real target point 
respectively on the axis of the real instrument, and a diameter of the at least one 
ring is dimensioned to achieve a pre-defined configuration together with the real 
instrument when the real instrument reaches the real target point. 
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